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Modelling tosk 


A channel has binary input x € {1,2} and real 
output у. 


Pl$g=1) = 


P(y|z) 


You see y. What is 2? 
Write P(x = 1|y) in the form 


1 
1 + exp (—something elegant) 


P(z = 1|y) = 


Information theory, pattern 
recognition, and neural networks 
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@Source coding (Data compression) 
2 Information content, entropy 
93 Typicality and the source coding theorem 
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Monte Carlo methods 


Simple Monte Carlo methods 
Importance sampling 
Rejection sampling 


Markov-chain Monte Carlo methods 
Metropolis method 
Gibbs sampling 
Slice sampling 


Reducing random-walk behaviour 
Hamiltonian Monte Carlo 


Overrelaxation 


Exact sampling 


Lecture 11 
Clustering 


Lecture 12 
Monte Carlo methods 


Lecture 13 
Advanced Monte Carlo methods 


Inferring parameters in science experiments 


A source emits particles; they decay at locations 
exponentially distributed with lengthscale A. De- 
cays can be observed only if the location falls in a 
window (Tmin, max) = (a,b) = (1, 20). 

N decays are observed at locations {21....,æn}. 
What is \? 


+ What about the possibility that the distribution is a mixture 
of two exponentials? 


jig ннн инно ана лелер P(Ostaltaui, tau2, ci..cN = 00000111) 


ML model with tuo tau's 


o 2 4 6 8 40 12 14 
Posterior distribution of the labels 


A channel has binary input x € {1,2} and real 
output у. 


Pl$g=1) = 


P(y|z) 


You see y. What is 2? 
Write P(x = 1|y) in the form 


1 
1 + exp (—something elegant) 


P(z = 1|y) = 


Clustering 


Clustering 


K means clustering 
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